Probabilistic Latent Semantic Analysis Research Articles

Use of Topic Analysis for Enhancing Healthcare Technologies

Nowadays, technology has played a vital role in the advancement of the healthcare sector. Various healthcare datasets are available on the web in the form of patents, scientific papers, articles, textual feedback, chatlogs, abstracts of papers, medical reports, and social media posts. It is a tedious task for the stakeholders to find hidden crucial knowledge on the discussed topic from this content, which if utilized optimally can lead to the rapid development of the healthcare sector. Topic analysis concepts are very effective in extracting meaningful topics from the data. Here, frequently applied Topic modeling methods -Latent Semantic Analysis, Probabilistic Latent Semantic Analysis, Latent Dirichlet Allocation, Correlated Topics Model, and Non-negative matrix factorization are surveyed along with their benefits and drawbacks. Insights on new innovative topic modeling techniques used in healthcare with their objective, opportunities, and challenges are provided, which can help the researchers for the enhancement of healthcare facilities.

Analysis of Suicide-related Tweets During the COVID-19 Pandemic

Background:: The COVID-19 virus started in 2019 and badly affected the different sectors of many countries around the world. Based on this, financial difficulties, loss of loved ones, sudden anger, relationships, family disputes, and psychological distress increased, and individuals were stalled from carrying out their lifestyle in a normal way, and some individuals were even motivated to commit suicide. Objective:: It is important to reduce the number of suicides and identify the reasons for this situation. Through this research, the focus is on identifying the main topics discussed relevant to suicides during the COVID-19 pandemic. Methods:: Individuals use Twitter, a social media platform, to share their ideas freely and publically. We collected 9750 primary data through Twitter API (Application Programming Interface). After preprocessing and feature extraction by TF-IDF (Term Frequency-Inverse Document Frequency), we applied the LDA (Latent Dirichlet Allocation) and Probabilistic Latent Semantic Analysis (PLSA) topic modeling algorithms to identify topics. Results:: Based on the LDA results, we extracted ten different topics under the three themes, such as the impact of COVID-19, human feelings, getting support, and having awareness. Intertopic Distance Map, Most Salient Terms, and Word Clouds Visualization are used to check the results. The coherence score and perplexing value are used to measure how interpretable the extracted topics are to humans. PLSA also extracted 25 topics with their probabilities, and Kullback–Leibler (KL) divergence was used to check the results. Conclusion:: We were able to gain insight into human emotions and the main motivations behind suicide attempts using the topics we extracted. Expert feedback proved that LDA results were better than PLSA. Based on that, we found the main impact of COVID-19 on human lives, how human feelings were changed positively and negatively during that period, what supporting and awareness methods people used, and what they preferred. The required measures can then be taken by those responsible authorities and individuals to prevent, reduce, and get ready for this kind of suicidal incident in the future.

Impact of subjective well-being on physical frailty in middle-aged and elderly Japanese with high social isolation.

Social isolation exacerbates physical frailty and is associated with subjective well-being. Even those with high levels of social isolation may have different health statuses depending on the type of isolation and their subjective well-being. However, the effect of subjective well-being on the relationship between social isolation and physical frailty remains unclear. This study examined whether the risk of physical frailty was the same for individuals with social isolation according to high and low subjective well-being. The study participants included 1,953 middle-aged Japanese adults aged 45 years and older. Physical frailty was assessed using a modified version of the Fried phenotype criteria. Probabilistic Latent Semantic Analysis was used to classify participants according to social isolation indicators. Subsequently, we focused on the groups with high social isolation and classified them according to whether their subjective well-being was high or low. Subjective well-being was evaluated using the Shiawase and Ikigai scales, which are concepts used in Japan. Finally, we used survival time analysis to examine the relationship between Shiawase or Ikigai and physical frailty in groups with high social isolation. The participants were classified into four groups based on their social isolation status. The physical frailty rate of the high social isolation class was 37.0%, which was significantly higher than that of the other classes. Survival time analysis revealed that among people with high social isolation, those with high Shiawase and Ikigai had a significantly lower risk of physical frailty than those with low Shiawase and Ikigai. All individuals with high social isolation are not at a high risk of physical frailty. The findings reveal that even those with high level of social isolation may have a lower risk of physical frailty if their subjective well-being is high. These results will contribute to promoting the prevention of frailty in middle-aged and older adults.

Supervised probabilistic latent semantic analysis with applications to controversy analysis of legislative bills

Probabilistic Latent Semantic Analysis (PLSA) is a fundamental text analysis technique that models each word in a document as a sample from a mixture of topics. PLSA is the precursor of probabilistic topic models including Latent Dirichlet Allocation (LDA). PLSA, LDA and their numerous extensions have been successfully applied to many text mining and retrieval tasks. One important extension of LDA is supervised LDA (sLDA), which distinguishes itself from most topic models in that it is supervised. However, to the best of our knowledge, no prior work extends PLSA in a similar manner sLDA extends LDA by jointly modeling the contents and the responses of documents. In this paper, we propose supervised PLSA (sPLSA) which can efficiently infer latent topics and their factorized response values from the contents and the responses of documents. The major challenge lies in estimating a document’s topic distribution which is a constrained probability that is dictated by both the content and the response of the document. To tackle this challenge, we introduce an auxiliary variable to transform the constrained optimization problem to an unconstrained optimization problem. This allows us to derive an efficient Expectation and Maximization (EM) algorithm for parameter estimation. Compared to sLDA, sPLSA converges much faster and requires less hyperparameter tuning, while performing similarly on topic modeling and better in response factorization. This makes sPLSA an appealing choice for latent response analysis such as ranking latent topics by their factorized response values. We apply the proposed sPLSA model to analyze the controversy of bills from the United States Congress. We demonstrate the effectiveness of our model by identifying contentious legislative issues.

Data distillation in computational pathology by choosing few representants of the original variance: A use case in ovarian cancer

In computational pathology, a typical Whole Slide Image may easily reach a size of 3–4 Gigabytes, while a database, with hundred of cases, might reach Terabytes. In this scenario, training any model is expensive and therefore the possibility of reducing the size of the training data is appealing. This paper presents a method to summarize the information of a data set, a function of the variance, by selecting the most informative samples. The first step of the whole strategy consists in projecting small image patches (100 × 100) to a feature space and discretize it with a simple k-means to obtain the feature space vocabulary. The groups obtained by the k-means are the vocabulary words and therefore any small patch is represented by the centroid of the group to which such patch is projected. A second step, a probabilistic Latent Semantic Analysis constructs groups of words known as topics by computing frequencies of words in documents, which are larger patches containing between 450 and 500 small patches. A third step collects the documents representing 80 % of the topic variance, their patches are assembled and a Singular Value Decomposition (SVD) is applied to these patches. The Data Distillation process chooses only the small patches belonging to the topics showing higher variance in the matrix of eigenvalues S from the SVD decomposition. The method efficacy was evaluated by comparing the performance of models trained either with 40 % of an actual ovarian cancer dataset selected using this method and the entire dataset without any selection. Results show the F-score obtained with these two sets was similar, about 0.87, with different classifiers, namely Support Vector Machine and a Multilayer Perceptron.

Revisiting Probabilistic Latent Semantic Analysis: Extensions, Challenges and Insights

This manuscript provides a comprehensive exploration of Probabilistic latent semantic analysis (PLSA), highlighting its strengths, drawbacks, and challenges. The PLSA, originally a tool for information retrieval, provides a probabilistic sense for a table of co-occurrences as a mixture of multinomial distributions spanned over a latent class variable and adjusted with the expectation–maximization algorithm. The distributional assumptions and the iterative nature lead to a rigid model, dividing enthusiasts and detractors. Those drawbacks have led to several reformulations: the extension of the method to normal data distributions and a non-parametric formulation obtained with the help of Non-negative matrix factorization (NMF) techniques. Furthermore, the combination of theoretical studies and programming techniques alleviates the computational problem, thus making the potential of the method explicit: its relation with the Singular value decomposition (SVD), which means that PLSA can be used to satisfactorily support other techniques, such as the construction of Fisher kernels, the probabilistic interpretation of Principal component analysis (PCA), Transfer learning (TL), and the training of neural networks, among others. We also present open questions as a practical and theoretical research window.

The method of grouping and classifying music curriculum teaching resources in the context of double reduction

Abstract A subsumption classification method is proposed to improve the classification accuracy of teaching resources in the music curriculum through a double reduction policy. Subject words are selected from high-similarity and high-frequency word sets, and a subject word tree is constructed using an automatic tree construction method based on a probabilistic latent semantic analysis algorithm. To complete the subsumption classification of teaching resources, an improved multi-graph kernel convolutional network is employed to group tree leaf nodes. According to the classification evaluation results, the recall, accuracy, and F1 values are 90%, 96.48%, and 88.81%, respectively, and the macro F1 value is as high as 81.59%. It can be seen that the method can effectively classify the teaching resources of music courses with the best effect of subsumption classification, which helps to improve the appropriateness and adequacy of teaching resources utilization.

Generating Markov Logic Networks Rulebase Based on Probabilistic Latent Semantics Analysis

Human Activity Recognition (HAR) has become a subject of concern and plays an important role in daily life. HAR uses sensor devices to collect user behavior data, obtain human activity information and identify them. Markov Logic Networks (MLN) are widely used in HAR as an effective combination of knowledge and data. MLN can solve the problems of complexity and uncertainty, and has good knowledge expression ability. However, MLN structure learning is relatively weak and requires a lot of computing and storage resources. Essentially, the MLN structure is derived from sensor data in the current scene. Assuming that the sensor data can be effectively sliced and the sliced data can be converted into semantic rules, MLN structure can be obtained. To this end, we propose a rulebase building scheme based on probabilistic latent semantic analysis to provide a semantic rulebase for MLN learning. Such a rulebase can reduce the time required for MLN structure learning. We apply the rulebase building scheme to single-person indoor activity recognition and prove that the scheme can effectively reduce the MLN learning time. In addition, we evaluate the parameters of the rulebase building scheme to check its stability.

Symbolic semantic design of industrial products based on Big data technology

Abstract Exploring the symbolic semantic design path of industrial products is to make industrial products more compatible with the diverse emotional needs of consumers. In this paper, starting from the sentiment analysis model, the PLSA-FSVM sentiment analysis method is constructed using a probabilistic latent potential semantic analysis method and support vector machine based on the Fisher kernel. The method’s validity is verified for comparative experiments and sentiment word frequency analysis evaluation. From the comparison experiments, the ten-fold cross-average precision and recall of PLSA-FSVM were 89.18% and 88.35%, respectively, 4.15% and 2.59% higher than PLSA-SVM. From the sentiment word frequency analysis, the percentages of sentiment words such as atmosphere, practical, and worthy are 23.08%, 22.59%, and 24.72%, respectively. This shows that the PLSA-FSVM sentiment analysis method can effectively realize the sentiment analysis of industrial product evaluation, promote the symbolic semantic design to be more in line with consumers’ emotional needs, and then realize the symbolic design of industrial products to reach the meaning with shape and enjoy with meaning.

Text Study of Reader Magazine in the Context of Big Data

Abstract Exploring the textual emotional value of the Reader is to help readers understand the Reader’s embedded emotions in an all-around way. In this paper, two text analysis methods, latent semantic analysis and probabilistic latent semantic analysis are described, starting from the analysis model of text research, and the automatic text classification technique is illustrated. The principles of support vector machines are explained, the optimal decision function selection solution is performed using kernel functions, and a PLSA-SVM text analysis model is also constructed using the PLSA method jointly with SVM. The structural features of Reader magazine were analyzed and explained, and the PLSA-SVM text analysis model was used to analyze the data on the narrative features of Reader magazine. In terms of narrative themes, the main narrative themes of Reader magazine were distributed as affection and love, which increased by 16.84% and 17.42% from 2012 to 2021, respectively. In terms of narrative perspectives, the proportion of first-person and third-person narrative perspectives is comparable, increasing by 9.03% and 12.3% from 2012 to 2021, respectively. Thus, in the context of big data, the PLSA-SVM text analysis model can be used to analyze the text narrative features of Reader magazine effectively, and Reader magazine can use the analysis to optimize further the text content and issue texts that are attuned to readers’ emotions.

Integrating Text Classification into Topic Discovery Using Semantic Embedding Models

Topic discovery involves identifying the main ideas within large volumes of textual data. It indicates recurring topics in documents, providing an overview of the text. Current topic discovery models receive the text, with or without pre-processing, including stop word removal, text cleaning, and normalization (lowercase conversion). A topic discovery process that receives general domain text with or without processing generates general topics. General topics do not offer detailed overviews of the input text, and manual text categorization is tedious and time-consuming. Extracting topics from text with an automatic classification task is necessary to generate specific topics enriched with top words that maintain semantic relationships among them. Therefore, this paper presents an approach that integrates text classification for topic discovery from large amounts of English textual data, such as 20-Newsgroups and Reuters Corpora. We rely on integrating automatic text classification before the topic discovery process to obtain specific topics for each class with relevant semantic relationships between top words. Text classification performs a word analysis that makes up a document to decide what class or category to identify; then, the proposed integration provides latent and specific topics depicted by top words with high coherence from each obtained class. Text classification accomplishes this with a convolutional neural network (CNN), incorporating an embedding model based on semantic relationships. Topic discovery over categorized text is realized with latent Dirichlet analysis (LDA), probabilistic latent semantic analysis (PLSA), and latent semantic analysis (LSA) algorithms. An evaluation process for topic discovery over categorized text was performed based on the normalized topic coherence metric. The 20-Newsgroups corpus was classified, and twenty topics with the ten top words were identified for each class. The normalized topic coherence obtained was 0.1723 with LDA, 0.1622 with LSA, and 0.1716 with PLSA. The Reuters Corpus was also classified, and twenty and fifty topics were identified. A normalized topic coherence of 0.1441 was achieved when applying the LDA algorithm, obtaining 20 topics for each class; with LSA, the coherence was 0.1360, and with PLSA, it was 0.1436.

Probabilistic inference on the factors promoting park use by machine learning using panel data from 2014 to 2020 in Tokyo, Japan

The health risks associated with confinement and physical inactivity during the COVID-19 pandemic continue to be a concern in urban areas. Access to urban green spaces is expected to contribute to increased physical activity. However, park use is the outcome of complex interactions among environmental, social, and individual factors. This study aimed to infer probabilistically factors promoting park use through hypothesis-generating research using two-point panel data analysis and machine learning before and after the COVID-19 epidemic in 2014 and 2020. We performed probabilistic inference using a structural model for probabilistic latent semantic analysis (pLSA) that combined pLSA and Bayesian network. The result indicated a slight but significant decrease in the frequency of park use between 2014 and 2020. However, the impact of COVID-19 on living condition and lifestyle was not the largest factor that influenced the frequency of park use in 2020 after COVID-19. Instead, we found that the presence of family members and social interaction with neighbors influenced the largest determinants of park use. The best social prescriptions for realistically and rapidly eliminating health disparities during the COVID-19 epidemic would be the implementation of interventions that simultaneously induce social capital improvement and park use.

Identifying meaningful subpopulation segments among older public assistance recipients: a mixed methods study to develop tailor-made health and welfare interventions

BackgroundPublic assistance recipients have diverse and complex needs for health and social support in addition to financial support. Segmentation, which means dividing the population into subgroups (segments) with similar sociodemographic characteristics, is a useful approach for allocating support resources to the targeted segments. Clustering is a commonly used statistical method of segmentation in a data-driven marketing approach. This explanatory sequential mixed methods study applied a clustering technique, aiming to identify segments among older public assistance recipients quantitatively, and assess the meaningfulness of the identified segments in consultation and support activities for older recipients qualitatively.MethodsWe identified the segments of older recipients in two municipalities using probabilistic latent semantic analysis, a machine learning-based soft clustering method. Semi-structured interviews were subsequently conducted with caseworkers to ask whether the identified segments could be meaningful for them in practice and to provide a reason if they could not think of any older recipients from the segment.ResultsA total of 3,165 older people on public assistance were included in the analysis. Five distinct segments of older recipients were identified for each sex from 1,483 men and 1,682 women. The qualitative findings suggested most of identified segments reflected older recipients in practice, especially two of them: female Cluster 1 (facility residents aged over 85 years with disability/psychiatric disorder), and female Cluster 2 (workers). Some caseworkers, however, did not recall older recipients in practice when working with certain segments.ConclusionsA clustering technique can be useful to identify the meaningful segments among older recipients and can potentially discover previously unrecognized segments that may not emerge through regular consultation practices followed by caseworkers. Future research should investigate whether tailored support interventions for these identified segments are effective.

Development and maturity of co-word thematic clusters: the field of linked data

PurposeThis research aimed to visualize and analyze the co-word network and thematic clusters of the intellectual structure in the field of linked data during 1900–2021.Design/methodology/approachThis applied research employed a descriptive and analytical method, scientometric indicators, co-word techniques, and social network analysis. VOSviewer, SPSS, Python programming, and UCINet software were used for data analysis and network structure visualization.FindingsThe top ranks of the Web of Science (WOS) subject categorization belonged to various fields of computer science. Besides, the USA was the most prolific country. The keyword ontology had the highest frequency of co-occurrence. Ontology and semantic were the most frequent co-word pairs. In terms of the network structure, nine major topic clusters were identified based on co-occurrence, and 29 thematic clusters were identified based on hierarchical clustering. Comparisons between the two clustering techniques indicated that three clusters, namely semantic bioinformatics, knowledge representation, and semantic tools were in common. The most mature and mainstream thematic clusters were natural language processing techniques to boost modeling and visualization, context-aware knowledge discovery, probabilistic latent semantic analysis (PLSA), semantic tools, latent semantic indexing, web ontology language (OWL) syntax, and ontology-based deep learning.Originality/valueThis study adopted various techniques such as co-word analysis, social network analysis network structure visualization, and hierarchical clustering to represent a suitable, visual, methodical, and comprehensive perspective into linked data.

Extracting marketing information from product reviews: a comparative study of latent semantic analysis and probabilistic latent semantic analysis

Extracting marketing information from product reviews: a comparative study of latent semantic analysis and probabilistic latent semantic analysis

Short text topic modelling using local and global word-context semantic correlation.

Nowadays, people use short text to portray their opinions on platforms of social media such as Twitter, Facebook, and YouTube, as well as on e-commerce websites such as Amazon and Flipkart to share their commercial purchasing experiences. Every day, billions of short texts are created worldwide in tweets, tags, keywords, search queries etc. However, this short text possesses inadequate contextual information, which can be ambiguous, sparse, noisy, remains a major challenge. State-of-the-art strategies of topic modeling such as Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis are not suitable as it contains a limited number of words in a single document. This work proposes a new model named G_SeaNMF (Gensim_SeaNMF) to improve the word-context semantic relationship by using local and global word embedding techniques. Word embeddings learned from a large corpus provide general semantic and syntactic information about words; it can guide topic modeling for short text collections as supporting information for sparse co-occurrence patterns. In the proposed model, SeaNMF (Semantics-assisted Non-negative Matrix Factorization) is incorporated with word2vec model of Gensim library to strengthen the word's semantic relationship. In this article, a short text topic modeling techniques based on DMM (Dirichlet Multinomial Mixture), self-aggregation and global word co-occurrence were explored. These are evaluated using different measures to gauge cluster coherence on real-world datasets such as Search Snippet, Biomedicine, Pascal Flickr, Tweet and TagMyNews. Empirical evaluation shows that a combination of local and global word embedding provides more appropriate words under each topic with improved outcomes.

Use of Neural Topic Models in conjunction with Word Embeddings to extract meaningful topics from short texts

Unsupervised machine learning is utilized as a part of the process of topic modeling to discover dormant topics hidden within a large number of documents. The topic model can help with the comprehension, organization, and summarization of large amounts of text. Additionally, it can assist with the discovery of hidden topics that vary across different texts in a corpus. Traditional topic models like pLSA (probabilistic latent semantic analysis) and LDA suffer performance loss when applied to short-text analysis caused by the lack of word co-occurrence information in each short text. One technique being developed to solve this problem is pre-trained word embedding (PWE) with an external corpus used with topic models. These techniques are being developed to perform interpretable topic modeling on short texts. Deep neural networks (DNN) and deep generative models have recently advanced, allowing neural topic models (NTM) to achieve flexibility and efficiency in topic modeling. There have been few studies on neural-topic models with pre-trained word embedding for producing significant topics from short texts. An extensive study with five NTMs was accomplished to test the efficacy of additional PWE in generating comprehensible topics through experiments with different datasets in Arabic and French concerning Moroccan news published on Facebook pages. Several metrics, including topic coherence and topic diversity, are utilized in the process of evaluating the extracted topics. Our research shows that the topic coherence of short texts can be significantly improved using a word embedding with an external corpus.

Text-based experiment retrieval in genomic databases

With the growing number of genomic data in public repositories, efficient search methodologies have become a basic need to reach the relevant genomic data. However, this need cannot be fulfilled with the current repositories because they offer a limited search option which is a lexical matching of textual descriptions or metadata of the experiments. This technique is insufficient to get the required information needed to detect similarities between experiments within a large data collection. Due to the limitation of the existing repositories, in this study, we develop a text-based experiment retrieval framework by using both lexical and semantic similarity approaches to find similarities between experiments, and their retrieval performance was compared. This study is the first attempt to use text-driven semantic analysis approaches for developing a retrieval framework for experiments. An empirical study was conducted on a large textual description of Arabidopsis microarray experiments from the Gene Expression Omnibus database. In the proposed model, Jaccard similarity was used as a lexical similarity approach; Latent Semantic Analysis, Probabilistic Latent Semantic Analysis and Latent Dirichlet allocation were used as semantic similarity approaches to detect similarities between the textual descriptions of the experiments. According to the experimental results, relevant experiments can be retrieved successfully by text-driven semantic similarity approaches compared with the lexical similarity approach.

Inkjet ink classification and source prediction based on direct analysis in real-time mass spectrometry (DART-MS) via mass imaging and convolutional neural network (CNN)

This paper presents mass spectrometry imaging (MSI) as an intuitive method of showing the differences between inks, and provides a novel method of ink source prediction based on direct analysis in real-time mass spectrometry (DART-MS) spectral analysis. The ink database comprised 106 types of ink from three brands (Canon, Epson, and HP) with high market share. High-dimensionality reduction methods, such as principal component analysis (PCA), non-negative matrix factorization (NMF), and probabilistic latent semantic analysis (pLSA), were used to reduce the dimensions of the mass spectral data. Characteristics and statistical descriptions of the mass spectra of each ink are given in this paper. Then, MSI was used to visualize the results after PCA, NMF, and pLSA. Finally, a convolutional neural network (CNN) was applied to the data after high-dimensionality reductions, which helped predict the ink source. Results showed efficient and excellent performance of machine-learning analysis in ink source prediction using one pixel in MSI (100% for black, magenta, and yellow inks; 99.6% for cyan ink). In addition, a blind test was used to validate the performance of the CNN model, with results showing that the more pixels used, the more accurate the results. Therefore, we strongly recommend wide-area detection of ink to ensure accurate results.

Аддитивная регуляризация при тематическом моделировании текстов сообществ онлайновых социальных сетей

The task of modeling communities (groups) of users in social media is relevant in the framework of information support for decision-making at different levels of government. For automated extraction of the meaning of textual and related information, topic modeling methods are used. This article presents the experience of improving the results of social networks communities topic modeling using the Additive Regularization for Topic Modeling (ARTM). The improvement of the results is achieved through the use of basic regularizers available in the open-source software BigARTM. Topic models obtained using regularized ARTM are compared with topic models obtained by Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis. The experiments were carried out on a dataset, containing preprocessed texts of posts from communities of an online social network. In particular, the quality of topic models in terms of coherence, the purity of topics, and the sparsity of the distribution matrices are compared. Disadvantages of coherence metrics for assessing the quality of topic models obtained using the ARTM method are discussed. Additional metrics are proposed that can be used for assessing the quality of topic models. Conclusions are drawn about the suitability of the ARTM approach for modeling communities of online social networks. The results of this work can be applied in the development of information and analytical systems for supporting the management of regional development.